y 

r 



GUO 11-7-3-8 

v 



BANDWIDTH GUARANTEED PROVISIONING IN NETWORK-BASED MOBILE 
VIRTUAL PRIVATE NETWORK (VPN) SERVICES 

CROSS-REFERENCE 

5 This patent application is a continuation-in-part of and claims priority to 

commonly owned U.S. patent application, serial number 10/374,940, filed February 
26, 2003, the entire contents of which are incorporated by reference herein. 

FIELD OF INVENTION 

10 The present invention relates to virtual private network (VPN) services. 

More specifically, the present invention relates to a service for providing 
connectivity for mobile devises utilizing VPN services. 

DESCRIPTION OF THE BACKGROUND ART f 

15 A Virtual Private Network (VPN) is a cost effective and secure way of 

i 

extending enterprise network resources over a shared public data network. Most 
popular uses of VPNs are to interconnect multiple geographically dispersed sites of 
an enterprise (known as intranet/extranet VPN) and to provide remote users 
access to the enterprise resources (known as remote access VPN). In particular, a 

20 virtual private network (VPN) is an overlay network that uses the public network to 
carry data traffic between corporate sites and users, maintaining privacy through 
the use of tunneling protocols and security procedures. 

In the network-based VPN model, an intranet/extranet VPN is created by 
interconnecting Customer Premise Equipments (CPE) of the enterprise to one or 

25 more VPN-aware network elements provisioned for the enterprise customer. A 
remote access VPN is created by tunneling the remote user's connection to a 
VPN-aware network element provisioned for the enterprise customer that the user 
belongs to. The VPN-aware network element then tunnels the connection to the 
appropriate CPE using tunnel concatenation. One such VPN-aware network 

30 element is a service switch called the IP Services Gateway (IPSG). An IPSG can 
be provisioned to serve a number of enterprise VPN customers each with a 
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number of end users. 

The basic method of setting up a VPN from a user or a site to secure 
enterprise resources is to set up a secure data connection between them over the 
underlying insecure shared network. An IPSG usually sets up two secure tunnels, 
5 one from the user/site to the IPSG itself, and the other from the IPSG to the 
enterprise. The IPSG is also responsible for maintaining separate and independent 
security associations with both ends, namely the user/site and the enterprise. The 
data flows end-to-end through the concatenated tunnel via the IPSG. Note that in 
a network-based VPN model, the network does not simply act as a conduit, but 

10 enables the VPN service. Moreover, the IPSG can enable other value-added 
services from the tunnel concatenation points. Examples include better QoS 
guarantees for VPN tunnels, service differentiation among users, offloading of 
Internet traffic from the enterprise intranet, and the like. 

The IPSG provisioning process creates virtual instances of routing 

15 mechanisms for each of the customers facilitated in the IPSG. In one possible 
implementation, each instance may be a separate (virtual) router running customer 
specific routing algorithms. In other implementations, each instance could be 
distinct customer specific route entries in the partitioned routing table. As such, 
each routing instance requires a considerable amount of computing resources. 

20 Further, since all the instances share the common resources of the IPSG, the 
number of VPN customers that can be provisioned on an IPSG is limited. There is 
a similar restriction on the number of tunnels an IPSG can support. Moreover, due 
to physical resource constraints, configuring an IPSG with increased number of 
provisions reduces the number of tunnels that can be handled, and vice versa. 

25 IPSG provisioning per customer is usually carried out statically because of the 
complexity of the process and IPSG provisioning is not changed frequently. 

At present, remote access VPNs are mostly limited to end users connecting 
to the enterprise from remote locations using wireline access like dial-up, DSL, and 
Cable-Modem lines. With the emergence of high-speed wireless data services in 

30 2.5G and 3G wireless technologies, VPN usage from mobile nodes (that is, mobile 
VPN services) is growing exponentially. 
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In order to enable mobile data services, a network service provider (NSP) 
installs wireless access devices at the edge of its network. Radio-to-packet 
network gateways (i.e., Mobile Access Points (MAPs)) connect the access devices 
to the data network. To set up a data session, a mobile end user (hereinafter 
5 termed a "mobile node" (MN)) must first connect to a MAP, which then routes the 
session towards the destination CPE through an appropriately provisioned IPSG. 
A mobile data session originating from a MN to a MAP, and then routed through an 
IPSG to the enterprise CPE, is the basis of a network-based mobile VPN service. 
Currently, the IPSG and MAP are collocated in the network, where an IPSG/MAP 

10 performs radio to packet network gateway functions to terminate the MN's 
connection, as well as conducting other IPSG specific functions. 

FIG. 1 depicts a high-level block diagram of a prior art mobile IP network 
100. In such a scenario the MN is not free to choose an IPSG, rather its data 
sessions are anchored to the IPSG serving the MN's current roaming region. Klote 

15 that the VPN service can be initiated only after the MN has started the 'data 
session. 

The exemplary network 100 comprises a backbone network 102, such as 
the Internet, a plurality of enterprise networks 120i through 120 r (collectively 
enterprise networks 120), and a network service provider (NSP) 101. The 

20 enterprise networks 120 each include at least one customer premise equipment 
(CPE) 122 and a plurality of mobile nodes (MN) 13d through 130 m collectively 
MNs 130). In this example, there are three VPN customers A, B and C, each with 
a corresponding intranet site 120. Customer A has two CPEs 122-m and 122 12 , 
while customers B and C each have one CPE 122 2 i and 122^. 

25 The NSP 101 comprises a network service provider access network 104 

having a plurality of IPSGs 106i through 106 q (collectively IPSGs 106). As shown 
in FIG. 1, IPSGi IO61 is provisioned for customer A 120^ IPSG 2 106 2 is 
provisioned for customers B and C 120 2 and 120 r (where r illustratively equals 3), 
IPSG3 IO63 is provisioned for all three customers A, B, and C, J \2Q- i , 120 2 , and 120 3 

30 and so on. This implies that IPSG1 IO61 has a routing instance for customer A, 
and a security association with CPE A i 122n and CPE^ 122 12 . The security 
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association is used to securely tunnel packets between IPSGi IO61 and the CPEs 
for customer A (that is, they have a pre-established secure tunnel). Similar 
associations hold for other IPSGs as well. In an instance where an MN 13d 
belonging to customer A roams into the region served by IPSG1 IO61, the MN 
5 successfully initiates a data session with IPSG1 IO61. Thereafter, the MN requests 
a VPN connection to CPE A i 122n. The IPSG1 serves this request by constructing 
a secure tunnel between the MN 130i and IPSG1 106! and concatenating it with 
the pre-established tunnel illustratively between IPSG1 IO61 and CPE A i 122n. 

Afterwards, when this MN 130i roams into the region served by IPSG 2 106 2 , 

10 the data session is reestablished with IPSG 2 106 2 . However, when the MN 
requests for the VPN service, IPSG 2 106 2 cannot provide such VPN service, since 
IPSG 2 106 2 is not provisioned for customer A. That is, IPSG 2 106 2 is not logically 
connected to CPE A i 122-n in a secure fashion. Later on, when this MN roams into 
the region serviced by IPSG 3 106 3 , the data session again is reestablished with 

15 IPSG3 IO63. When the MN 13d requests for the VPN service, IPSG 3 106 3 is able 
to provide the VPN session, since IPSG 3 106 3 is provisioned for customer A. 

Presently, in one solution termed "uniform-provision", in addition to IPSG 
IPSGt 106i, IPSG 3 106 3i and IPSG 5 106 q = 5 , the NSP also provisions IPSG 2 106 2 
and IPSG4 IO64 for customer A. The first uniform-provision solution implies that 

20 every IPSG 106 in the network is provisioned for all the customers of the NSP. 
This is required because of the mobile nature of the users, that is, an MN 
belonging to any customer can roam into the region served by any IPSG 106 and 
request for service. Therefore, no IPSG 106 can a priori assume that it would 
serve only a subset of the customers. 

25 For example, suppose the NSP has "N" IPSGs and each can support at 

most "M" different provisions (recall that the number of VPN customers that can be 
provisioned per IPSG is limited). The total number of different provisions the NSP 
100 can provide is therefore M x N. However, under this solution each IPSG 106 
must be provisioned exactly the same way with every VPN customer. In practice, 

30 every VPN customer must be provisioned on every IPSG 106, and this limits the 
total number of supported VPN customers to merely M. Accordingly, this 
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connectivity solution does not scale with the number of subscribed VPN customers. 
This, however, is not a problem for non-mobile VPN services such as remote 
access VPNs from home and intranet/extranet VPNs, since the NSP knows a 
priori, which customers are going to connect to which IPSGs (due to their 
5 geographic locations) and can statically provision the IPSGs with only the relevant 
subset of VPN customers. 

A second solution, termed "tunnel-switching", allows an IPSG to be 
provisioned for a subset of customers. For example, IPSG 2 106 2 tunnels the MN's 
data session to an IPSG that is provisioned for customer A, such as IPSGi IO61. 
10 The tunnel-switching solution requires each IPSG 106 to be aware of the 

provisions made by other IPSGs, detect the identity of the MN, and tunnel switch 
the session to an appropriate IPSG 106. It is noted that in certain cases, this 
method results in using more than one tunnel to connect an MN 130 to the .. 

appropriately provisioned IPSG 106. I 

i 

15 The second tunnel-switching solution provisions each IPSG with a subset of 

VPN customers, and supports mobility through tunnel switching the MN's data 
sessions from the IPSG in the MN's roaming area to the appropriately provisioned 
IPSG. That is, the tunnel-switching solution maintains connectivity by using two or 
more tunnels to connect an end user to the appropriate IPSG 106. 

20 The tunnel-switching second solution for providing MN-IPSG-CPE 

connectivity does not scale, since in order to handle more VPN customers, the 
IPSGs must support more tunnels, which in turn will reduce the number of 
provisions that can be made per IPSG 106. Moreover, tunnel switching among 
IPSGs leads to undesirable redirection of connections (commonly known as "dog- 

25 legging") within the NSP's network, which results in an inefficient usage of network 
links. 

SUMMARY OF THE INVENTION 

The disadvantages heretofore associated with the prior art, are overcome by 
30 the present invention of a method and apparatus for optimally provisioning 

connectivity in network-based mobile virtual private network (VPN) services. The 
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apparatus includes provisioning each of a plurality of IP service gateways (IPSGs) 
to support virtual private network (VPN) tunneling between customer premise 
equipment of a subset of VPN customers and at least one mobile access point 
(MAP). The MAPs are geographically remote from the plurality of IPSGs, and each 
5 of the MAPs support VPN tunneling to mobile nodes of the subset of VPN 
customers. 

In one embodiment, a first method includes for each customer, selecting a 
subset of IPSGs to maximize total profit resulting from provisioning the customers 
on the selected IPSGs, wherein the total profit from all the customers comprises 

10 the sum of profits from each customer, where for each customer profit (G) equals 
weighted revenue less cost. The weighted revenue includes revenue and a 
relative weight factor Y on revenue compared to cost, where r allows a network 
service provider to adjust price based on cost of the customer. Further, the cost 
per customer comprises a total tunnel connection cost from the MAP to the CPE, 

15 and a current cost of provisioning an IPSG node, wherein the total tunnel 

connection cost comprises a dynamic tunnel connection cost between the MAP 
and the provisioned IPSG, and a static tunnel connection cost between the 
provisioned IPSG and the CPE. 

In a second embodiment, a method and virtual private network (VPN) 

20 system architecture is provided for providing bandwidth guaranteed provisioning in 
network-based mobile VPN services. The method and system architecture include 
identifying a set of VPN customers, at least one mobile access point (MAP) and at 
least one customer premise equipment (CPE) associated with each VPN customer, 
and at least one IP service gateway (IPSG) for facilitating VPN tunneling between 

25 a MAP and a CPE, wherein each MAP is geographically remote from each IPSG. 
A subset of IPSGs is selected to maximize total profit resulting from provisioning a 
subset of VPN customers on the selected IPSGs. Total profit from all the 
customers includes the sum of profits from each customer (/), where for each 
customer, profit ((J) equals weighted revenue ( r V 1 ) less cost (d) (U= r V'-C'), 
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wherein the cost per customer includes a total tunnel bandwidth cost (Cc) from the 
MAP to said CPE, and a cost (CV) of provisioning an IPSG node. 

BRIEF DESCRIPTION OF THE DRAWINGS 

5 The teachings of the present invention can be readily understood by 

considering the following detailed description in conjunction with the accompanying 
drawings, in which: 

FIG. 1 depicts a high-level block diagram of an exemplary prior art mobile IP 

network; 

10 FIG. 2 depicts a high-level block diagram of an exemplary mobile IP network 

of the present invention; 

FIG. 3 depicts a flow diagram of a method for providing virtual private 
network (VPN) services based on link costs; j 

FIG. 4 depicts a schematic diagram of a first undirected graph for a single 

15 customer; ' 

FIG. 5 depicts a flow diagram of a method suitable for selecting a subset of 
IP service gateways (IPSGs) to provision a single VPN customer in accordance 
with the method of FIG. 3; 

FIG. 6 depicts a schematic diagram of a second undirected graph for 
20 multiple customers; 

FIG. 7 depicts a flow diagram of a method for providing virtual private 
network (VPN) services based on bandwidth constraints; 

FIG. 8 depicts a flow diagram of a method suitable for selecting a subset of 
IP service gateways (IPSGs) to provision a single VPN customer based on 
25 bandwidth capacity in accordance with the method of FIG. 7; and 

FIG. 9 depicts a flow diagram of a method suitable for selecting a subset of 
IP service gateways (IPSGs) to provision multiple VPN customers based on 
bandwidth capacity in accordance with the method of FIG. 7. 

To facilitate understanding, identical reference numerals have been used, 
30 where possible, to designate identical elements that are common to the figures. 
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DETAILED DESCRIPTION OF THE INVENTION 

The present invention provides a method and apparatus for provisioning 
VPN-aware devices in an hierarchical network architecture for mobile virtual private 
networks (VPNs). The methods discussed herein take into account the cost of 

5 links over which VPN tunnels are established, the cost of establishing a tunnel, the 
cost of provisioning a VPN customer on a VPN-aware device, such as an IP 
service gateway (IPSG), and redundancy in IPSG provisioning for fault tolerance. 

FIG. 2 depicts a high-level block diagram of an exemplary mobile IP network 
200 of the present invention. The exemplary network 200 comprises a backbone 

10 network 202, such as the Internet, a plurality of enterprise networks 220i through 
220 r (collectively enterprise networks 220), a service provider 201 (e.g., network 
service provider (NSP)), and a plurality of mobile nodes (MN) 230i through 230 m 
collectively MNs 230). 

The enterprise networks 220 may be an intranet/extranet network, each 

15 having at least one customer premise equipment 222. In this example, there are 
three VPN customers A, B, and C, each- with a corresponding intranet site 220. 
Customer A has two CPEs 222n and 222 12 , while customers B and C each have 
one CPE 222 2 i and 222*. The CPE 222 typically includes a customer edge router, 
or Layer Two Tunneling Protocol (L2TP) network server, among other conventional 

20 customer network equipment. 

The service provider 201 includes a network service provider access 
network 204 having a plurality of IP service gateways (IPSGs) 206i through 206 q 
(collectively IPSGs 206) and a plurality of wireless access devices 208 positioned 
at the edge of the network 204, separate and apart from the IPSGs 206. That is, in 

25 order to enable mobile data services, the NSP 201 installs the wireless access 
devices 208 at the edge of its network 204. 

In one embodiment, the wireless access devices 208 are radio-to-packet 
network gateways, hereinafter termed Mobile Access Points (MAPs), which are 
used to connect the access devices to the data network. Accordingly, a packet 

30 data serving node (PDSN) in the CDMA 2000 architecture or a gateway GPRS 
support node/serving GPRS support node (GGSN/SGSN) in the UMTS 
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architecture may serve as the MAPs. To set up a data session, a mobile end user 
utilizing a mobile node (MN) 230 must first connect to a MAP 208, which then 
routes the session towards the destination CPE through an appropriately 
provisioned IPSG. Thus, a network-based mobile VPN service mobile data 
5 session originates from an MN 230 to a MAP 208, and is then routed through a 
particular IPSG 206 to the enterprise CPE 222. 

In the network-based VPN architecture of FIG. 2, the MAPs 208 are 
separately and hierarchically located from the IPSGs 206. In particular, a MAP 208 
serves a region, and all MNs 230 within that region, regardless of customer 
10 association, connect to the MAP 208 to initiate data sessions. Each IPSG 206 is 
statically provisioned for only a subset of the enterprise VPN customers. The 
subsets per IPSG 206 are chosen so that at least one IPSG is provisioned for each 
customer. j 

In the illustrative embodiment shown in FIG. 2, IPSGi 206i and I#SG 5 

i 

15 206 qss5 are provisioned for VPN customer A 220i, and IPSG 2 206 2 , IPSG 3 206 3 ', and 
IPSG 4 2O64 are provisioned for VPN customer B 220 2 . Similarly, IPSG 2 206 2 and 
IPSG5 2O65 are provisioned for VPN customer C 220r= 3 . Mobile traffic destined to 
customer A is directed by MAPs 208 to either IPSG1 206i or IPSG 5 206 5l 
depending on the location of the MN 230. Each IPSG 206 only needs to support a 

20 subset of the three VPN customers 220. 

An IPSG maintains the virtual routing instance and security association with 
each provisioned VPN customer 220. Each MAP 208 maintains a simple and fairly 
static list of customer-to-IPSG mappings. It is noted that the list changes only 
when a new customer 220 subscribes to the VPN service offered by the NSP, 

25 which is very infrequent. When an MN 230 requests a VPN connection to its 
respective CPE 222, the MAP 208 identifies the customer the MN belongs to, and 
routes and/or tunnel switches the connection to the appropriate IPSG 206 
provisioned for the customer. 

In particular, the MNs 230 are identified using, for example, conventional 

30 Network Access Identifiers (NAI) and/or Access Point Names (APN). A MAP 
extracts the NAI/APN of the MN 230 during connection setup time with the MN. 
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The MAP can then identify the destination CPE 222 directly from the NAI/APN, if 
there is only one CPE 222. If there is more than one CPE 222, the MAP can 
determine the MN's 230 preferred CPE 222 from an Authentication, Authorization 
and Accounting (AAA) Server of the service provider 201 . 
5 By illustration, in an instance where an MN 230i belonging to customer A 

roams into the region served by MAP^ 208-^, the MAP 208 n identifies the customer 
the MN 230i belongs to, and routes and/or tunnel switches the connection to the 
appropriate IPSG 206 provisioned for the customer. In this example, the 
connection is routed to IPSGi 206l The IPSd 206i serves this request by 

10 constructing a secure tunnel between the MN 230i and IPSG1 206i and 
concatenating it with the pre-established tunnel between IPSG1 206i and, 
illustratively, CPE A i 222n. 

If the MN 23d roams into a region served by MAP 2 206 2 , the connection is 
also routed to IPSG1 206i. Similarly, if the MN 23d roams into a region served by 

15 either MAP 3 206 3 or MAP 4 206 4 , the connection is also routed to IPSGi 206! or 
IPSG5 206q= 5 (connections not shown in FIG. 2). 

Accordingly, one aspect of the present invention is to separate mobility from 
services, where a MAP 208 deals with mobility of users, while an IPSG 206 offers 
VPN services. This is a natural division of functions because IPSGs are designed 

20 to support services for stationary locations, while the MAPs are designed to handle 
mobility by providing dynamic switching and routing. 

The above approach solves the scalability issues for the MN-IPSG-CPE 
connectivity problem. The scalability in provisioning is addressed by allocating a 
subset of VPN customers per IPSG, with at least one provision for every customer. 

25 In other words, this approach provisions a subset of IPSGs per customer. 
Compared with the existing architecture where each customer has to be pro- 
visioned on every IPSG, the hierarchical approach naturally offers improved 
scalability. The tunnel switching and associated dog-legging are taken care of by 
locating MAPs 208 separately and hierarchically with respect to IPSGs 206. In this 

30 hierarchical design, the MAPs 208 offer tunnel switching/traffic redirection 
functionalities. Therefore the MAPs 208 are able to separate intranet VPN traffic 
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from internet traffic, direct VPN traffic to the appropriate IPSGs 206, and direct 
internet traffic to appropriate internet proxies in the NSP network 204. This value- 
added Internet traffic offloading service effectively saves bandwidth for the NSP 
201 and its customers over the existing architecture where MAPs 208 and IPSGs 
206 are collocated. 

Thus, in order to establish the NSP's network connectivity, each VPN 
customer is mapped to a subset of IPSGs 206. One solution is to map/provision as 
many customers as possible on one IPSG 206, and then use a second IPSG 206 
only when the current one is full. However, this technique does not utilize the 
resources fully. That is, it creates hot spots and degrades the overall performance 
of the network. 

Alternatively, in an embodiment of the present invention, a subset of IPSGs 
206 is chosen in an optimal fashion for each customer, so that all the IPSGs 206 
are equally provisioned/utilized, while there is also room for inclusion of future j 
customers. Determining the best set of IPSGs 206 to provision for each customer 
includes various factors, such as the cost of links over which VPN tunnels are 
established, the cost of establishing a tunnel, the cost of provisioning a VPN 
customer on an IPSG 206, and redundancy in IPSG provisioning for fault 
tolerance. 

FIG. 3 depicts a flow diagram of a method 300 for providing virtual private 
network (VPN) services based on link costs. The method 300 begins at step 301, 
and proceeds to step 302, a network service provider (NSP) 201 strategically 
distributes a plurality of IPSGs 206 across various geographic regions, such as, for 
example, in various parts of a large city, across a state, and/or nationwide. At step 
304, the NSP 201 distributes a plurality of mobile access points (MAPs) 208 across 
the various geographic regions, such that the MAPs 208 are located separate and 
apart from the IPSGs 206. 

The method 300 then proceeds to step 306, where the number and location 
of network nodes are identified. When the NSP 201 deploys the nodes in the 
network, the number and location of each IPSG 206 and MAP 208 are identified. 
Further, the number of customers and their respective intranets 220 and CPEs 222 



250125 



11 



GUO 11-7-3-8 



are also identified. Also identified is the hop count between the nodes, such that 
an end-to-end hop count may be determined from a MN 230 to a CPE 222. Once 
the nodes and hop counts have been identified, the method 300 then proceeds to 
step 308. 

5 At step 308, the NSP 201 selectively provides connectivity between each 

customer 220 (i.e., CPE 222) and at least one IPSG 206. That is, a determination 
is made to resolve particular subsets of IPSGs 206 to be provisioned for a 
customer. Selecting a subset of the plurality of IPSGs 206 to serve each customer 
220 is based on a cost analysis algorithm, which is discussed below in further 
10 detail with respect to FIG. 5. At step 310, the NSP 201 selectively provides 
connectivity between each MAP 208 and at least one IPSG 206. Selection of the 
IPSGs 206 to support the MAPs 208 is also based on cost analysis, which is also 
discussed below in further detail with respect to FIG. 5. The method 300 then 
proceeds to step 312. 

15 At step 312, the selected IPSGs 206 are provisioned with virtual routing 

instances and security associations for the customer. At step 314, the provisioned 
IPSGs 206 are used to establish VPN tunnels to the corresponding CPEs 222 of 
the customer. In particular, VPN tunnels may be established from the mobile 
nodes 230 to their respective CPE 122 via a MAP 208 serving the mobile node 230 

20 and a customer specific IPSG 206. The method 300 then proceeds to step 399, 
where the users participate in a VPN session and the method 300 ends. 

FIG. 4 depicts a schematic diagram of a first undirected graph 400 for a 
single customer. In particular, FIG. 4 depicts a schematic diagram of a first 
undirected graph 400 having a set of nodes and a set of links, and is suitable for 

25 understanding method 300 of FIG. 3. The network illustratively comprises T MAPs 
208, "/' IPSGs 206, and u k' CPEs 222 for a given customer, respectively denoted 
by p h q jt and r k . In the exemplary graph 400 of FIG. 4, the network 400 comprises 
two MAPs 208! and 208 2 denoted pi and p 2 , three IPSGs 206i, 206 2 , and 206 3 
denoted q-,, q 2 , and q 3 in the network 400, and a single customer 220 having two 

30 CPEs 222i and 222 2 denoted n and r 2 for the customer. Furthermore, a plurality of 
mobile nodes 230! through 230 m (where m is an integer greater than 1) is 
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illustratively shown coupled to the MAPs 208. Specifically, MNi 23d through MN 3 
230 3 have connectivity to MAP Pi 208i, while MN 4 230 4 and MN m 230 m have 
connectivity to MAP P 2 208 2 . 

It is noted that multiple VPN customers are considered in a batch and the 
5 best set of IPSGs are determined to provision for the batch that will maximize the 
profit. For each customer, the CPEs in the customer's intranet, and all of the 
IPSGs and MAPs in the NSP network are considered. 

The network of IPSGs 206, MAPs 208, and the customer's CPEs 222 is 
modeled as an undirected graph G = (V, E), where V is the set of nodes and E is 
10 the set of links. Graph nodes in V correspond to the CPEs 222, IPSGs 206, and 
MAPs 208 only. Graph links in E may be categorized as a link between a MAP 
208 and an IPSG 206 corresponds to the chosen path between the corresponding 
MAP and the IPSG, and a link between an IPSG 206 and a CPE 222 corresponds 

to the chosen path therebetween. A routing algorithm based on a particular roiiting 

i 

15 objective computes the chosen paths. The routing objective may be the shortest 
path based on hop counts, or the lowest-cost path based on the cost assigned to 
network links, both of which can be computed by open shortest path first (OSPF). 
The routing objective may also be a traffic-engineered path such as an ATM VC or 
an MPLS Label Switched Path. In the hierarchical architecture of the present 

20 invention, traffic flows from MNs 230 to the CPEs 222 through the MAPs 208 and 
IPSGs 206. Therefore, only links between them are considered. 

Referring to FIG. 3, at step 308, a subset of the IPSGs 206 is selected for 
each customer. The establishment of a VPN tunnel over a physical network link 
incurs a certain cost associated with the link. The cost of a link between two nodes 

25 in the graph then becomes the computed cost of the VPN tunnel between the 
corresponding network nodes. In practice, depending on the requirement of the 
VPN customer, the link cost may be the number of hops in the underlying physical 
network or a fraction of the bandwidth capacity of the physical links, among other 
link cost measuring techniques. For purposes of understanding the invention, link 

30 costs are discussed in terms of an optimal connectivity between the MAPs and the 
CPEs (i.e., number of hops), as opposed to computing link costs using bandwidth 
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capacity of a physical link. 

Since only one VPN tunnel is established between an IPSG and a CPE for 
the same customer, the cost of a link from an IPSG to a CPE is considered only 
once for each customer. For example, referring to FIG. 4, IPSG q 3 206 3 may have 
5 two tunnels formed from MAPs p* and p 2 208i and 208 2 via respective links p^ 3 
and P2Q3. However, only one shared tunnel is utilized between the IPSG q 3 206 3 
and CPE r 2 222 2 for those MNs connecting to CPE r 2 222 2 . 

FIG. 5 depicts a flow diagram of a method 500 suitable for selecting a 
subset of IP service gateways (IPSGs) to provision a VPN customer in accordance 

10 with the method of FIG. 3. In particular, method 500 is suitable for providing step 
308 of FIG. 3. Method 500 starts at step 501, and proceeds to steps 502, where 
predetermined network parameters are identified. In particular, the predetermined 
network parameters include a set of all MAPs (P), a set of all IPSGs (Q), a set of all 
customer CPE (R), the cost (c,y) of sending traffic from each MAP 208 to each 

15 IPSG 206, the cost (d jk ) of sending traffic from each IPSG 206 to each CPE 222, 
and the current cost (fj) for using an IPSG node <j) 206. 

At step 504, dynamic tunnel connection costs (C C i) are formulated as 
between the MAPs (p, of FIG. 4) and IPSGs (q, of FIG. 4). Further, at step 506, 
static tunnel connection costs (C C 2) are formulated as between the IPSGs (g, of 

20 FIG. 4) and the CPEs (r* of FIG. 4). 

In particular, connection cost may be considered in terms of VPN tunnels. 
For every session from a user of a customer to a CPE, a VPN tunnel is established 
from a MAP to an IPSG. The VPN tunnel from a MAP to an IPSG is referred to as 
a "dynamic tunnel", since the VPN tunnel is typically established by a user "on-the- 

25 fly". However, the traffic from the IPSG to the CPE will be aggregated over one 
tunnel, termed a "static tunnel". In this instance, the cost from an IPSG to a CPE is 
included in the overall connection cost only once. For purposes of clarity and 
understanding the invention, optimization and selection of the IPSGs is formulated 
for a single customer, and then generalized for multiple customers. 
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A service providers profits may be maximized by selecting optimal IPSGs to 
provision a given VPN customer. It is noted that profit (G= Y R-C) is the difference 
between weighted revenue (Y R) and cost (C), where revenue (R) for a customer is 
a fixed value if the customer can be provisioned and Y is the relative weight on 
5 revenue compared to cost. 

The cost has several components, and as discussed above, determining 
the best set of IPSGs to provision each customer includes factoring in the cost of 
links over which VPN tunnels are established, the cost of establishing a tunnel, the 
cost of provisioning a VPN customer on an IPSG, and redundancy in IPSG 
10 provisioning for fault tolerance. In other words, for every MAP / in P and every 
CPE k in R, an IPSG j in Q is selected to establish a unique dynamic tunnel 
between / and y, and a shared static tunnel between j and /c, such that the profit is 
maximized. I 
Referring to FIG. 4, P is the set of all MAPs 208, Q is the set of all IPSGs 
15 206, and R is the set of all CPEs 222 for a customer. The binary variable Xi jk e {0,1} 
denotes whether a dynamic tunnel between node / e P and node j e Q is used for 
the traffic from MAP / to CPE k e R. The binary variable z jk e {0,1} denotes 
whether a shared static tunnel from IPSG j to CPE k is established. Here the cost 
of sending traffic from node / to node j is c,y, and the cost of sending traffic from 
20 node j to node k is d Jk . 

For a single customer, the dynamic tunnel connection cost (which is 
illustratively the hop count cost between the MAPs and the IPSGs) is Cci = 
^CyXp . Similarly, the static tunnel connection cost (which is illustratively the 

iePJeQ y keR 

hop count cost between the IPSGs and the CPEs) is Cc2 = X^y z y* ■ 

25 At step 508, the total tunnel connection cost (C c ) is formulated. The total 

connection cost is the sum of the dynamic tunnel connection cost and the static 
tunnel connection cost Cc = Cci + G>Cc2, where B is the relative weight on the static 
tunnel connection cost. Factors influencing the relative weight G on the static 
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tunnel connection cost include the cost of transporting data over core network over 
the cost over access network. 

At step 510, the current cost C v of provisioning a IPSG node G) is 
formulated. The binary variable y, e {0,1} is 1 if IPSG j is provisioned for the 
customer to send traffic to at least one of its CPEs, and it is 0 otherwise. The 
parameter fj is illustratively used as the current cost of using IPSG node/ For a 
given customer, at most one provision is considered at any IPSG. Therefore fj has 
a fixed value when only one customer is considered at a time, and the provisioning 

cost isC v = 

At step 512, the total cost for the customer is formulated. In particular, the 
total cost is C = C c + aC v where a is the relative weight on the provision cost. 
Factors influencing the relative weight a on the provision cost include the 
importance of provision costs over connection costs for the network service 
provider. 

At step 514, the profit is formulated. In particular, the profit is G = Y R - C. 
For simplicity, revenue R =1 . Therefore, the profit "G" for provisioning the 
customer is G = Y - C, where Y is the relative weight on revenue compared to total 
cost. The weighting factor ^essentially allows the network service provider to 
adjust price based on the total cost for the customer. 

At step 516, given parameters c ijf d jk f jf or, fi, and Y , binary variables x ijk , z jk 
and y, are determined as the solution to the optimization problem formulation 
expressed as: 



maxG =y-C, 



0) 



where 



C = (C cl +/3C C2 )+aC y 



(2) 




(3) 



\iePJeQMR jeQMR ) J*Q 



x, k e {0,1}, Vi e Pyj <=Q,VkeR 



(4) 



250125 



16 



GUO 11-7-3-8 



z Jk e{0,l},Y/e2,V*€/? (5) 

y,e{O,l},Y/e0 (6) 

^x iJk = l,V/e/»,V*e* (7) 

j*Q 

x IJk < z jk , Vi e P, V/ eQ,VkeR (8) 

5 z jk <yj,VjeQ,VkeR (9) 



It is noted that equation (3) is an expanded version of equation (2). It is 
further noted that equation (7) specifies that exactly one link out of a MAP is 
chosen to go to one CPE, thereby implying that traffic from a MAP to a CPE is sent 
to only one IPSG. Equation (8) specifies a condition that only one tunnel is 
10 established between an IPSG and a CPE, even if traffic from multiple MAPs are 
going through the IPSG to reach the CPE. That is: 

=Uf 2X* >O,V/€0,V*e* (10) | 

z jk =0 otherwise. (11) 
Equations (10) and (11) are equivalent to condition (8), since z jk is in the objective 
15 function G, and when x ijk = 0, VieP. to maximize G, z jk = 0 must be chosen. 

The condition expressed in equation (9) specifies that even if an IPSG is 
provisioned to send traffic to more than one CPE, for the purpose of computing 
provision cost, it should be considered as only one provision. That is, 

y J =U1^z Jk >0,VjeQ (12) 

20 y>j = 0 otherwise. (13) 

Equations (12) and (13) are equivalent to the condition expressed in 
equation (9), since yj is in the objective function G, and when z jk = 0,V keR, to 
maximize G, y y = 0 must be chosen. At step 518, the method 500 ends. 

Once the provisioning costs are determined, the profit G for provisioning a 
25 customer with a particular subset of IPSGs may be computed. Specifically, profit 
equals revenues less provisioning costs G = Z-C. In other words, the 
connectivity between the mobile node 230 and CPE 222 may be optimized, since 
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the sum of the costs between the nodes (i.e., hop count) and the cost of 
provisioning IPSGs is minimized by provisioning a particular subset of IPSG 206 
for a customer 220. 

In the multiple customer case, the sum of the profit for each customer is 
5 maximized, where the profit for each customer is calculated exactly the same way 
as in the single customer case discussed above. All MAPs 208 and IPSGs 206 in 
the network are shared among all customers. However, each customer has its 
distinct set of CPEs. 

In the single customer case, the provision cost fj at each IPSG j has a fixed 

10 value, and an IPSG that has reached its provision capacity is not considered, 

which is equivalent to setting fj = °°. When multiple customers are considered, fj is 
assigned a fixed value for all customers provisioned on IPSG j\ however, because 
multiple customers can be provisioned at each IPSG, care must be taken to ensure 
that the number of customers provisioned does not exceed the provision capacity 

15 of each IPSG. Moreover, when multiple customers are considered at the same 
time, not every customer should be provisioned in the network. Priorities should be 
given to customers providing maximum profit. There are two cases where a 
customer is rejected. One case is when there is no more provision capacity left on 
any IPSG in the network, the other case is when provisioning this customer results 

20 in negative profit, meaning a loss. Essentially to maximize the total profit a subset 
of the customers are provisioned. The rest of the customers are rejected because 
either the provision capacity is reached or they produce a loss instead of profit. 

The optimization problem for multiple VPN customers can be described as 
follows. Let 7 be the set of VPN customers to consider and |r|= Z. Let P be the 

25 set of all MAPs, Q be the set of all IPSGs, and R be the set of all CPEs for all 
customers where R={Ri,R2, Ri, -^ 3 Ri} and R f is the set of CPEs for customer 
/ e T . Let vJ be the binary variable specifying if customer / should be provisioned 
in the network. For each customer / provisioned, every node / in P and every node 
k in f?/, choose an IPSG node j in Q, to establish a unique tunnel between / and j\ 
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and a shared tunnel between j and k, such that the total profit for all customers is 
maximized. Needless to say, for a customer not provisioned, the cost is 0. 

FIG. 6 depicts a schematic diagram of a second undirected graph 600 for 
multiple customers. FIG. 6 is the same as FIG. 4, except that two customers 220 
5 are illustratively shown, each having two CPEs 222. In particular, the network 600 
illustratively comprises V MAPs 208, "/ IPSGs 206, and 4 7c" CPEs 222 for a given 
customer, respectively denoted by p h q Jf and r k . In the exemplary graph 600 of 
FIG. 6, the network 600 comprises two MAPs 208<i and 208 2 denoted Pi and p 2 , 
three IPSGs 206i, 206 2 , and 206 3 denoted q 1t g 2 , and c/ 3 in the network 400, and a 

10 two customers 220^ and 220 2 . Each customer illustratively has two CPEs, such as 
CPE 222n and 222 12 denoted r ni and r 12 for a first customer 220-,, and CPE 222 21 
and 222 22 denoted r 2 t and r 22 for a second customer 220 2 . Furthermore, a plurality 
of mobile nodes 230 m is illustratively shown coupled to the MAPs 208. Specifically, 
MNi 230! through MN 3 230 3 have connectivity to MAP P t 208i, while MN 4 230 4 l 

1 5 and MN m 230 m have connectivity to MAP P 2 208 2 . I 

For customer / e 7\ we denote by the binary variable x* ijk g {0,1} whether a 
tunnel between node i g P and node j g Q is used for the traffic from MAP / to 
CPE fee R r We use binary variable z Jk g {0,1} to denote whether a shared tunnel 

from IPSG j to CPE k is established. The cost of sending traffic from node i to 
20 node j is Cy. Notice that the cost is the same for all customers, and therefore index 
/ is not needed. The cost of sending traffic from node j to node k is dj k . 

The binary variable y) g {0,1} is 1 if IPSG j is provisioned for customer I to 

send traffic to at least one of its CPEs, and it is 0 otherwise. We use parameter 
Pcap as the maximum number of customers that can be provisioned on each IPSG, 
25 and parameter f s as the cost for customer / to use node / As long as the provision 
capacity of IPSG / has not been reached, the provision cost for each customer is 
the same, and therefore index / is not needed. 

For a single customer / g T under consideration, the dynamic tunnel 
connection cost (which is the cost from MAPs to IPSGs) is C l cl = ^ iePJeQkeR ^ 0 x l ok . 
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The shared static tunnel connection cost (which is the cost from IPSGs to MAPs) is 
C' C2 = keR ^a z % * The tota ' connection cost is therefore, C* c = C^, + fiC l C2 , where 
P is the relative weight on static tunnel connection cost. The provisioning cost for 
customer / is C l v = "jL, Q f jy*j . Thus, the total cost for customer / is C 1 = C l c + aC^ 

5 where a is the relative weight on the provision cost. The revenue for each 

customer provisioned is assumed to be the same. Accordingly, both the revenue 
and cost are zero for each customer not provisioned. The profit is therefore 
G 1 = yw 1 -C 1 where y is the relative weight on revenue compared to cost. The 
optimization problem formulation can then be specified as 
10 maxG=S^ 1 04) 



where 



G l =yw l - C'yieT (15) 
C'=(c^+jK£ 2 )+oCj (16) 
C 1 = ( £ c,4+>9 E ^) + «Z^ 07) 

15 w'eWD.V/eT (18) 

xj, e {0,1}V/ e T, V/ e P,VJ e 0,V* e tf, (1 9) 

z l Jk e{0,\}VleTyj^Q,\/keR, (20) 

>>;<= {o,i> v/e r.vy €fi (21) 

IX* =w',V/e7\V/eP,V/fcert, (22) 

20 < z' jk ,\fl e 7\Vz e P, Vj e 0,V* e tf, (23) 

z' jk < yJ.V/ e T, Vy e g, V* e tf, (24) 
2>j < >^,y/ e 2 (25) 

It is noted that equation (17) is an expanded version of equation (16). 
Compared with the formulation for a single customer, condition (25) is added to 
25 specify that the total number of provisions on each IPSG j cannot exceed its 
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capacity Pcap. Moreover, a new binary variable vJ is introduced to specify if a 
customer is provisioned, and Condition (22) is modified to specify that only when a 
customer is provisioned, exactly one link out of a MAP is chosen to go to one CPE 
for this customer, otherwise no link out of any MAP is chosen and no IPSG is 
5 provisioned. 

In order to solve the integer-programming problem discussed above, 
connection costs c,j, d Jk and provision cost fj need to be assigned appropriate 
values. The cost computation can be adapted to fit the NSP's design objectives. 
This makes the above formulation quite general and can be used for different 

10 scenarios in addition to guaranteeing connectivity for VPN customers. 

Connection cost is a function of the parameters that the NSP wants to 
control. A NSP 204 may need to satisfy a special requirement from a VPN 
customer 220, such that the users of this customer are not switched to a rerpote 
lightly loaded IPSG 206, even if that reduces the total cost for the NSP 204. I For 

15 example, an MN 230 on the east coast trying to access corporate intranet on the 
east coast should not be switched to an IPSG on the west coast even if the total 
cost is minimized with this solution. To take the constraint into account, we restrict 
the number of hops allowed from a MAP to a CPE and the link cost of the graph is 
modified as: 

20 Cij = «, if dj > LI m ax, Vi g P, and Vj e Q (26) 

d jk = oo, if d ik > L2 max , Vj g Q, Vk g R, and V/ g T (27) 
where L1 ma * and L2 max are the maximum number of hops allowed for the tunnel 
between a MAP 208 and a IPSG 206 and the tunnel between the IPSG 208 and a 
CPE 222, respectively. 

25 When a single customer is considered at a time, the provision cost may be 

optionally set to reflect the existing number of provisions at each IPSG. For 
example, the provisioning cost fj= cap/availj, where cap; is the capacity of IPSG j 
and avalji is the number of available provisions left. This cost assignment will result 
in even distribution of the number of provisions per IPSG across all IPSGs 206. 

30 However, when multiple customers are considered at the same time, the 
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provision cost for different customers has to be the same to be a valid input to the 
integer-programming program. Without loss of generality, we set $=1 for IPSG /for 
all customers. 

The cost computation phase accounts for customer specific requirements. 
5 After the cost computation phase, conventional integer programming packages 
(e.g., LPSOLVE and CPLEX) may be used to solve the IPSG selection problem. 

Fault tolerance may be provided to ensure that if a tunneling IPSG fails, at 
least second IPSG is available to provide redundancy. In one embodiment, a 
minimum bound is placed on the replication. In order to provide fault tolerance, for 
10 every customer, each traffic session from a MAP to a CPE should have the option 
of going through N > 1 IPSGs. In case N-1 IPSGs fail, traffic sessions can still be 
established using the functioning IPSG. The only modification to the formulation 
provided in condition (22) without fault tolerance consideration, is to substitute 
Condition (22) with 

15 5>!» =Mv',V/€7\Vi€i>,V*e/?, (28) 

Condition (28) specifies for each customer / e T that is provisioned, there 
must be N connections established between a MAP 208 and a CPE 222 each 
going through a separate IPSG 206. Because each pair of MAP and CPE requires 
the use of a set of N IPSGs, and these IPSG sets can overlap, therefore the total 
20 number of IPSGs used for customer / is greater than or equal to N. In other words, 
this formulation specifies the minimum number of IPSGs provisioned for each 
customer. 

In a second embodiment, an exact bound is placed on the replication. 
Another way to consider fault tolerance is to require that each customer can only 
25 use exactly N IPSGs for all its connections. This would require one more condition 
to be added to the formulation as follows: 

£j/=Mv',V/€r (29) 

Condition (29) specifies exactly N IPSG nodes can be used for all the 
connections for a provisioned customer /. 
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Accordingly, a hierarchical architecture using two network elements, namely 
the MAPs 208 and IPSGs 206, has been illustratively shown to provide mobile 
VPN services. In order to optimally use the network elements, several costs have 
been identified, which influence the designing of the network. In particular, in one 
5 embodiment, the IPSGs are provisioned for mobile VPN customers in order to 
minimize the total connection cost of links over which VPN tunnels are established, 
as well as the cost of provisioning IPSGs for each customer. 

That is, multiple customers of the NSP may be optimally provisioned onto 
different IPSGs 206 in order to maximize the profit for an NSP that provides 
10 mobile-VPN services. Such optimization takes into account the cost of links over 
which the VPN sessions are established and the cost of provisioning customers on 
the IPSGs 206. 

In a second embodiment of the invention, the VPN customers may be ■. 

optimally provisioned onto different IPSGs 206 in a bandwidth limited network. I In 

i 

15 particular, using the same architecture described above with respect to FIGS. 1-6, 
maximizing the NSPs profit may be determined based on bandwidth constraints of 
the network. Such determination of maximizing the NSPs profitability takes into 
account the bandwidth requirements of the customers, the cost of allocating the 
required bandwidth, the capacity restraints of the links, and the cost of provisioning. 

20 the customers on the IPSGs, as discussed below with respect to FIGS. 7-9. 

Specifically, mobile users belonging to enterprise customers of the NSP 
initiate data connectivity with a MAP 208 that covers the region where that user is 
located. These users then initiate VPN sessions (i.e., tunnels) with a specific CPE 
222 that is located at one of the customer locations. As discussed above, these 

25 sessions are initiated via dynamic VPN tunnels by a user when secure connectivity 
to the enterprise is required, and torn down when the user does not have this 
requirement anymore. These VPN session requests are forwarded by the 
corresponding MAPs 208 to an IPSG 206 that has been provisioned for that 
customer. The IPSGs 206 terminate these VPN sessions, and forward traffic 

30 from/to these sessions to the appropriate CPE 222 over static VPN tunnels (i.e., 
long-lived pre-initiated VPN sessions) from the IPSGs 206 to the CPEs 222. This 
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means, for every VPN session from a MN 230 of a customer to a CPE 222, a 
dynamic tunnel is established by the MN 230 through the corresponding MAP 208 
to an IPSG 206. However, the traffic from the IPSG 206 to the CPE 222 will be 
aggregated over one static tunnel. In between, the IPSGs provide value-added 
5 services to the traffic forwarded between pairs of dynamic and static tunnels. As 
discussed above with respect to the hierarchical architecture, each customer is 
provisioned only within a subset of IPSGs, and only those IPSGs can provide 
Mobile-VPN services to the users of the customer. 

In this scenario, an IPSG 206 provisioning problem in bandwidth 

10 constrained networks may be defined as follows. Consider a specific Mobile-VPN 
customer. The NSP realizes a certain amount of revenue by providing Mobile-VPN 
services to the users of this customer. However, there is also a cost associated 
with providing this service. With respect to this customer, the problem to be solved 
for by the NSP is (a) select a subset of IPSGs to provision for this customer, and 

1 5 (b) for each MAP 208, determine how much of the traffic that arrives at the MAP 
208 from users of this customer should be redirected to each of the IPSGs 206 in 
this subset, under the constraint that the network bandwidth capacity is limited. 
The objective is to find a solution to problems (a) and (b) such that the profit 
(revenue minus cost) obtained by the NSP by providing Mobile-VPN service is 

20 maximized. The more general problem is to find a solution such that the total profit 
obtained by the NSP by providing Mobile-VPN services to all customers is 
maximized. 

For purposes of understanding the invention, it is assumed that the NSP 
realizes a fixed amount of revenue from a customer by providing Mobile-VPN 

25 service. The cost associated with providing the service can be enumerated as (i) a 
fixed cost associated with provisioning a customer on an IPSG 206, which is 
referred to as the provision cost; and (ii) a variable cost associated with sending 
traffic from/to a MAP 208 to/from an IPSG 206 over dynamic tunnels and from/to 
an IPSG 206 to/from a CPE 222 over static tunnels, which is a function of the 

30 amount of traffic (in units/sec, e.g., Mbits/sec) sent (i.e., bandwidth costs). 
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Logical links that connect MAPs 208 to IPSGs 206 and IPSGs 206 to the 
CPEs 222 have a certain capacity (in units/sec (e.g., Mbits/sec)). The traffic that is 
sent by a MAP to an IPSG and similarly by an IPSG to a CPE cannot exceed the 
underlying link capacity. A logical link is assumed to represent a set of physical 
5 links that provide a path between the corresponding pair of elements (i.e., nodes). 
Additionally, each IPSG is limited to a certain number of provisions that it can 
handle in terms of the number of customers. This is because of the resources that 
an IPSG has to reserve for each of the customers. 

It is further assumed that the MAPs and IPSGs are already deployed by the 
10 NSP and their numbers and locations are chosen during the network planning 

process. It is also assumed that the NSP provides network connectivity so that any 
MAP 208 can reach any IPSG 206, and any IPSG 206 can reach any CPE 222. 
The capacity on the (logical) link between a MAP 208 and an IPSG 206, and ... 

similarly between an IPSG 206 and a CPE 222 is normally determined by the I 

i 

15 bottleneck physical link on the path between the pairs of elements. It is assumed 
that this capacity information is available to the NSP a priori. 

Additionally, it is assumed that MNs 230 are identified using the popular 
methods of Network Access Identifier (NAI) and/or Access Point Name (APN). A 
MAP 208 extracts the NAI/APN of the MN during data connection setup time with 

20 the MN. From the NAI/APN, it can then identify the destination CPE 222, if there is 
only one CPE. If there is more than one CPE 222, the MAP 208 can determine the 
MN's preferred CPE from an Authentication, Authorization and Accounting (AAA) 
server (not shown). When a MAP 208 redirects a VPN session request from a MN 
230 to an IPSG 206, a dynamic tunnel is established between the MN and the 

25 IPSG. At this point, the IPSG 206 is aware of the CPE 222 that the MN 230 wants 
to connect to. From this point on, whenever traffic is received on the dynamic 
tunnel from the MN 230, the IPSG 206 will forward it through the static tunnel 
towards that CPE 222. Thus, data connectivity is established end-to-end between 
a MN 230 and a CPE 222. In steady state, it is assumed that the NSP has an 

30 estimate of how much bandwidth is needed to service all the users (MNs) of a 
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customer that establish data connectivity at a MAP and send traffic to a specific 
CPE. 

FIG. 7 depicts a flow diagram of a method 700 for providing virtual private 
network (VPN) services based on bandwidth constraints. FIG. 7 should be viewed 
5 in conjunction with FIGS. 2, 4, and 6. The method 700 begins at step 701, and 
proceeds to step 702, a network service provider (NSP) 201 strategically 
distributes a plurality of IPSGs 206 across various geographic regions, such as, for 
example, in various parts of a large city, across a state, and/or nationwide. 

When an enterprise customer signs up for the VPN service, it provides the 

10 NSP the location of its CPEs. The NSP can process customer requests in two 
ways. In the simple approach, it can process customer requests one at a time, as 
discussed below with respect to FIG. 8. This approach will achieve local optimal, 
however, it cannot guarantee global optimal. That is, it cannot guarantee all 
customers are optimally provisioned to maximize the profit for NSP. The other 

15 approach is to consider multiple customers at a time, as discussed below with 
respect to FIG. 9. Since the formulation with multiple customers achieves global 
optimal, and the single customer formulation is considered a special case of the 
multi-customer approach. 

At step 704, the NSP 201 distributes a plurality of mobile access points 

20 (MAPs) 208 across the various geographic regions, such that the MAPs 208 are 
located separate and apart (i.e., remote) from the IPSGs 206, as discussed above 
with respect to steps 302 and 304 of FIG. 3. 

The method 700 then proceeds to step 706, where the number and location 
of network nodes are identified. When the NSP 201 deploys the nodes in the 

25 network, the number and location of each IPSG 206 and MAP 208 are identified, 
as well as the number of customers and their respective intranets 220 and CPEs 
222 are identified. Also identified is the bandwidth capacity between the nodes, 
such that the end-to-end bandwidth capacity may be determined from a MN 230 to 
a CPE 222. It is noted that the connectivity between nodes may be the shortest 

30 path based on hop counts, or the lowest-cost path based on the cost assigned to 
network links, both of which can be computed by open shortest path first (OSPF). 
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Once the nodes and bandwidth capacity between the nodes have been identified, 
the method 700 proceeds to step 308. 

At step 708, the NSP 201 selectively provides connectivity between each 
customer 220 (i.e., CPE 222) and at least one IPSG 206. That is, a determination 
5 is made to resolve particular subsets of IPSGs 206 to be provisioned for a 
particular customer. Selecting a subset of the plurality of IPSGs 206 to serve 
selected customers 220 is based on a cost analysis algorithm, which is discussed 
below in further detail with respect to FIG. 8. Once connectivity is provided 
between each customer CPE 222 and the at least one IPSG 206, the method 700 
10 proceeds to step 710. At step 710, the NSP 201 selectively provides connectivity 
between each MAP 208 and at least one IPSG 206. Selection of the IPSGs 206 to 
support the MAPs 208 is also based on cost analysis, which is also discussed 
below in further detail with respect to FIG. 8. The method 700 then proceeds to 
step 712. 

15 At step 712, the selected IPSGs 206 are provisioned with virtual routing 

instances and security associations for the customer. At step 714, the provisioned 
IPSGs 206 are used to establish VPN tunnels to the corresponding CPEs 222 of 
the customer. In particular, VPN tunnels may be established from the mobile 
nodes 230 to their respective CPE 222 via a MAP 208 serving the mobile node 230 

20 and a customer specific IPSG 206. The method 700 then proceeds to step 799, 
where method 700 ends, and the users may participate in a VPN session. 

Referring to FIG. 4, the network of IPSGs 206, MAPs 208, and the 
customer's CPEs 222 is again modeled as an undirected graph G= (V,E.) where V 
is the set of nodes and E is the set of links, as discussed above. In particular, FIG. 

25 4 shows an example graph model for a customer. There are two MAPs, pi and p 2 , 
three IPSGs, qi, q2, and q 3 in the network, and two CPEs n and r 2 for the 
customer. Graph nodes in V correspond to CPEs 222, IPSGs 206, and MAPs 208. 
Graph links in E fall in the following two categories: (1) a link between a MAP and 
an IPSG corresponds to a logical link between the MAP and the IPSG, and (2) a 

30 link between an IPSG 206 and a CPE 222 corresponds to a logical link between 
the IPSG and the CPE. As mentioned above, a logical link represents a set of 
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physical links that form a path between the corresponding elements. This path 
could be the shortest path based on hop counts, or the lowest-cost path based on 
the cost assigned to network links, both of which can be computed by OSPF. It 
could also be a traffic-engineered path such as an ATM, VC, an MPLS Label 
5 Switched Path, and the like. 

The capacity of a link is the capacity of the bottleneck physical link on the 
path that corresponds to this link. In the hierarchical architecture, traffic flows from 
MNs 230 to CPEs 222 through the MAPs 208 and IPSGs 206. Therefore, only 
links between MAPs 208 and IPSGs 206 and between IPSGs 206 and CPEs 222 

10 are considered in the model. It is assumed that there are I MAPs, J IPSGs in the 
network and K CPEs for a given customer, denoted by p h qj, and r ki respectively. 

Based on the above model, a solution to the IPSG provisioning problem 
may be determined by considering a set of Mobile-VPN customers and (a) 
computing the best set of IPSGs 206 to provision for each of the customers, and 

1 5 (b) determining how the flow of traffic (in terms of units/sec) for a customer, 

received at a MAP 208 destined to a specific CPE, will be split and sent to the set 
of IPSGs 206 on which the customer has been provisioned. The solution is first 
formulated for a single customer as discussed below with respect to FIG. 8. The 
solution is then extended to incorporate multiple customers, as discussed below 

20 with respect to FIG. 9. 

Referring to FIG. 7, at step 708, a subset of the IPSGs 206 is selected for 
each customer. The establishment of a VPN tunnel over a physical network link 
incurs a certain cost associated with the link. In this second embodiment, the cost 
of a link between two nodes in the graph then becomes the computed cost of the 

25 VPN tunnel between the corresponding network nodes. In this second 

embodiment, the link costs are associated with a fraction of the bandwidth capacity 
of the physical links, as opposed to the number of hops in the underlying physical 
network, as discussed with respect to the first embodiment of FIGS. 1-6. Thus, 
the link costs are discussed in terms of bandwidth capacity of a physical link. 

30 Since only one VPN tunnel is established between an IPSG and a CPE for 

the same customer, the cost of a link from an IPSG to a CPE is considered only 
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once for each customer. For example, referring to FIG. 4, IPSG q 3 2O63 may have 
two tunnels formed from MAPs p 1 208i and p 2 208 2 via respective links p^q 3 and 
P2<73- However, only one shared tunnel is utilized between the IPSG q 3 206 3 and 
CPE r 2 222 2 for those MNs connecting to CPE r 2 222 2 . 
5 FIG. 8 depicts a flow diagram of a method 800 suitable for selecting a 

subset of IP service gateways (IPSGs) to provision a single VPN customer based 
on bandwidth capacity in accordance with the method 700 of FIG. 7. FIG. 8 should 
be viewed in conjunction with FIG. 4. Method 800 starts at step 801, and proceeds 
to step 802, where predetermined network parameters are identified. In particular, 
10 the predetermined network parameters include a set of all MAPs (P), a set of all 
IPSGs (Q), a set of all customer CPE (R), the bandwidth capacity between MAP i 
and IPSG j (g,y), the bandwidth capacity between IPSG j and CPE k (h Jk ) t the 
bandwidth requirement between MAP i and CPE k (b*), the unit bandwidth cost on 

the link between MAP i to IPSG j (a,y), the unit bandwidth cost on the link betwefen 

1 

15 IPSG j to CPE k (e,*), current cost (fj) for using an IPSG node. I 
Additionally at step 802, integer and binary variables are identified. The 
integer variable (s ijk ) is used to specify the amount of traffic from MAP i to CPE k 
that is directed through IPSG j, and the binary variable y 7 e {0,1} specifies whether 
or not IPSG j is provisioned for the customer to send traffic to at least one of its 

20 CPEs. 

In particular, Let P be the set of all MAPs, Q be the set of all IPSGs, and R 
be the set of all CPEs for the customer as shown in FIG. 4. The integer variable s ijk 
is used to specify the amount of traffic from MAP i e P to CPE k g R that is 
directed through IPSG j g Q. It is assumed that the capacity on the link between 

25 MAP i g P and IPSG j e Q is gy units/sec, the capacity of the link between IPSG j 
g Q and CPE k g R is h jk units/sec, and the bandwidth requirement for traffic from 
MAP i g P to CPE k g R is b ik units/sec, where units/sec may illustratively be 
Mbits/second or Mbytes/second. It is also assumed that the unit bandwidth cost (1 
unit/sec) on the link from MAP i to IPSG j is a,y, and the unit bandwidth cost on the 

30 link from IPSG j to CPE k is e*. 
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At step 804, the cost (c,y) of sending traffic from each MAP 208 to each IPSG 
206 is formulated for a single customer. In particular, the bandwidth requirement 
for traffic from MAP i e P to IPSG j e Q is ^s iJk . Therefore, the bandwidth cost 

between MAP i and IPSG j is c u = a v ^s^ . 

keR 

5 Similarly, at step 806, the cost (d jk ) of sending traffic from each IPSG 206 to 

each CPE 222, and the current cost (fj) for using an IPSG node (j) 206 is 
formulated for a single customer. In particular, the bandwidth requirement for 
traffic from IPSG j e Q to CPE k e R is ^s iJk . Therefore, the bandwidth cost 

between IPSG j and CPE k is d Jk = e Jk ^s iJk . 

ieP 

10 At step 808, dynamic tunnel bandwidth costs (Cci) are formulated as 

between the MAPs (p, of FIG. 4) and IPSGs (q, of FIG. 4). Specifically, the 
bandwidth cost of dynamic tunnels is C C1 = = ^a 0 s iJk . Further, at step 

810, static tunnel bandwidth costs (C C 2) are formulated as between the IPSGs (q/ 
of FIG. 4) and the CPEs (r* of FIG. 4). Specifically, the bandwidth cost of static 
15 tunnels is C C2 = J^d Jk = X>,*^ . 

jgQMR iePJeQMR 

A service provider's profits may be maximized by selecting optimal IPSGs to 
provision a given VPN customer. It is noted that profit (L/= Y R-C) is the difference 
between weighted revenue (Y R) and cost (C), where revenue (R) for a customer is 
a fixed value if the customer can be provisioned and Y is the relative weight on 

20 revenue compared to cost. 

The total cost has several components, and as discussed above, such as 
determining the best set of IPSGs to provision each customer, which includes 
factoring in the cost of links in terms of bandwidth over which VPN tunnels are 
established, the cost of establishing a tunnel, the cost of provisioning a VPN 

25 customer on an IPSG, and redundancy in IPSG provisioning for fault tolerance. In 
other words, for every MAP / in P and every CPE k in R, an IPSG j in Q is selected 
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to establish a unique dynamic tunnel between / and j t and a shared static tunnel 
between j and k, such that the profit is maximized. 

At step 812, the total tunnel bandwidth cost (Cc) is formulated. The total 
bandwidth cost is the sum of the dynamic tunnel bandwidth cost and the static 
5 tunnel bandwidth cost Cc = Cci + SCc2, where Q> is the relative weight on the 

bandwidth cost of the static tunnel. Factors influencing the relative weight Q> on the 
static tunnel bandwidth cost include the cost of transporting data over core network 
over the cost over access network. This is because the connection from IPSG to 
CPE is over the core network and the connection between MAP and IPSG is over 
1 0 the access network. 

At step 814, the current cost Cv of provisioning a IPSG node (j) is 
formulated. The binary variable y ; e {0,1} is 1 if IPSG j is provisioned for the 
customer to send traffic to at least one of its CPEs, and it is 0 otherwise. The .. 
parameter fj is illustratively used as the current cost of using IPSG node /. For £ 

i 

15 given customer, at most one provision is considered at any IPSG. Therefore // has 
a fixed value when only one customer is considered at a time, and the provisioning 

cost isC v = . 

J*Q 

At step 816, the total cost for the customer is formulated. In particular, the 
total cost is C = Cc + aC v , where a is the relative weight on the provision cost. 
20 Factors influencing the relative weight a on the provision cost include the 
importance of provision costs over bandwidth costs for the network service 
provider. 

At step 818, the profit is formulated. In particular, the profit is U = / V - C. 
For simplicity, revenue V =1. Therefore, the profit "U" for provisioning the customer 
25 is U = y - C, where y is the relative weight on revenue compared to total cost. 
The weighting factor ^essentially allows the network service provider to adjust 
price based on the total cost for the customer. 
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At step 820, given parameters g ijt h jkt b ik a,y, e jk f Jf a, &, and Y , integer 
variable s iJk and binary variable yj are determined as the solution to the optimization 
problem formulation expressed as: 





maxU = y - C , 




(30} 


5 


where 








C = 


(C cl +/3C C2 )+ocC y 




(31) 
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(32) 
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keR 


fk <g iJ9 VieP,\/j<EQ 


(38) 






(39) 


15 


s m < yj b ik , V/ e P, V/ e 0, V* e 


(40) 



It is noted that equation (32) is an expanded version of equation (31). It is 
further noted that equation (35) specifies that traffic for the customer at MAP i 
destined to CPE k could be split and forwarded through multiple IPSGs. 
Conditions (36) and (37) define the bandwidth cost of dynamic tunnel from MAP i 
20 to IPSG j, and static tunnel from IPSG j to CPE k, respectively. Conditions (38) 
and (39) specify the total bandwidth restrictions for dynamic tunnels and static 
tunnels, respectively. Further, equation (40) specifies a condition that even if an 
IPSG is provisioned to send traffic to more than one CPE, for the purpose of 
computing provision cost, it should be considered as only one provision. That is: 
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>v=uf 2>**>o,Y/ee (41) 

y } = 0 otherwise. (42) 
Equations (41) and (42) are equivalent to condition (40), since 
buc * *i,* v/ e />,./ e e * , and when >0, as long as y,= 1 , condition (40) is 
5 satisfied. In addition, since y, is in the objective function U, and when s ijk =0, VieP, 
t/k(P to maximize profit U, yj = 0 must be chosen. At step 899, the method 800 
ends. 

Once the provisioning costs are determined, the profit U for provisioning a 
customer with a particular subset of IPSGs may be computed. Specifically, profit 
10 equals revenues less provisioning costs U = X - C . In other words, the connectivity 
between the mobile node 230 and CPE 222 may be optimized, since the sum of 
the costs between the nodes (i.e., bandwidth constraints) and the cost of 
provisioning IPSGs is minimized by provisioning a particular subset of IPSG,206 
for a customer 220. 

15 In the multiple customer case, the sum of the profit for each customer is 

maximized, where the profit for each customer is calculated exactly the same way 
as in the single customer case discussed above. All MAPs 208 and IPSGs 206 in 
the network are shared among all customers. However, each customer has its 
distinct set of CPEs. 

20 In the single customer case, the provision cost fj at each IPSGy has a fixed 

value, and an IPSG that has reached its provision capacity is not considered, 
which is equivalent to setting fj = «>. When multiple customers are considered, fj is 
assigned a fixed value for all customers provisioned on IPSGy, however, because 
multiple customers can be provisioned at each IPSG, care must be taken to ensure 

25 that the number of customers provisioned does not exceed the provision capacity 
(PCAP) of each IPSG. Moreover, when multiple customers are considered at the 
same time, not every customer should be provisioned in the network. Priorities 
should be given to customers providing maximum profit. There are three instances 
where a customer is rejected. One case is when there is no more provision 
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capacity left on any IPSG in the network. Another instance occurs when the 
bandwidth requirement of the customer from one or more of the MAPs exceeds the 
network capacity, while the third other case occurs when provisioning a customer 
results in negative profit (i.e., a loss). Essentially, a subset of the customers is 
5 provisioned to maximize the total profit. The rest of the customers are rejected 
because either the provision capacity is reached or they produce a loss instead of 
profit. 

Referring to FIG. 6, two customers 220 each having two CPEs 222 are 
shown, as opposed to the single customer shown in FIG. 4. The network 600 

10 illustratively comprises V MAPs 208, "/ IPSGs 206, and "Jc" CPEs 222 for a given 
customer, respectively denoted by p„ q,, and r k . Recall that the network 600 
comprises two MAPs 208i and 208 2 denoted and p 2 , three IPSGs 206! , 206 2l 
and 206 3 denoted q 1f q 2t and q 3 \n the network 600, and two customers 220i and 
220 2 . Each customer illustratively has two CPEs, such as CPE 222n and 222 12 

15 denoted rn and r 12 for a first customer 220 1f and CPE 222 21 and 222 22 denoted r 21 
and r 22 for a second customer 220 2 . Furthermore, a plurality of mobile nodes 230 m 
is illustratively shown coupled to the MAPs 208. Specifically, MN1 230i through 
MN 3 230 3 have connectivity to MAP p 1 208V, while MN 4 230 4 and MN m 230 m have 
connectivity to MAP p 2 208 2 . For customer 1 , the two CPEs are rn 222nand r i2 

20 222 12 , and for customer 2, the two CPEs are r 2 i 222 21 and r 22 222 22 . 

FIG. 9 depicts a flow diagram of a method 900 suitable for selecting a 
subset of IP service gateways (IPSGs) to provision multiple VPN customers based 
oh bandwidth capacity in accordance with the method 700 of FIG. 7. FIG. 9 should 
be viewed in conjunction with FIG. 6. Method 900 starts at step 901, and proceeds 

25 to step 902, where predetermined network parameters and variables are identified. 

In particular, Let T be the set of mobile VPN customers to consider such that 
|r| = L , where L represents the number of VPN customers. Let P be the set of all 
MAPs, Q be the set of all IPSGs, and R be the set of all CPEs for all customers 
where R= {Ri, R 2 , Ru - Ri} and R/is the set of CPEs for customer / e 7\ as 

30 illustratively shown in FIG. 6. Let vJ be the binary variable specifying if customer / 
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should be provisioned in the network. The optimization problem formulation for 
multiple mobile VPN customers may be specified as, for each customer / 
provisioned, every node / in P and every node k in f?/, choose an IPSG node y in Q, 
to forward traffic through a unique (dynamic) tunnel between / and y, and a shared 
5 (static) tunnel between j and k t such that the total profit for all customers is 
maximized. Needless to say, for a customer not provisioned, the cost is 0. 

For customer /e T, an integer variable s'y* is used to specify the amount of 
traffic from MAP /e P to CPE ke R lt which is directed through IPSG ye Q. That is, 
s'jjk represents the amount of traffic from MAP i to CPE k that is directed through 
10 IPSG j. The parameters g$ and h) k represent the bandwidth capacity between MAP 
i and IPSG j, and the bandwidth capacity between IPSG j and CPE k, respectively. 
The parameter b 1 ,* represents the bandwidth requirement between MAP / and CPE 
k. It is assumed that the capacity on the link between MAP i e P and IPSG j e Q 

is gy units/sec, the capacity of the link between IPSG j e Q and CPE k e Ri is rfjk 

i 

15 units/sec, and the bandwidth requirement for traffic from MAP i e P to CPE k e R| 
is b* ik units/sec, where units/sec may illustratively be Mbits/second or 
Mbytes/second. It is also assumed that the unit bandwidth cost (1 unit/sec) on the 
link from MAP i to IPSG j is a,j, the unit bandwidth cost on the link from IPSG j to 
CPE k is e'yjf, and the current cost for using an IPSG node is fj. 

20 The binary variable y l e {0,1} is 1 if IPSG j is provisioned for customer / to 

forward traffic to at least one of its CPEs, and is 0 if the IPSG j is not provisioned 
for customer // The binary variable w* e {0,1} is 1 if customer / is provisioned in the 
network, otherwise zero. Pqap represents the maximum number of customers that 
can be provisioned on each IPSG, and fj represents the cost for customer i to use 

25 node j. The provision cost for each customer at an IPSG is assumed to be the 
same. The method 900 then proceeds to step 904. 

At step 904, the cost (d$ of sending traffic along the dynamic tunnel from 
each MAP 208 to each IPSG 206 is determined for each customer /. In particular, 
the bandwidth requirement for traffic from MAP i e P to IPSG j e Q is ][V/y* . 
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Therefore, the bandwidth cost for each customer / between MAP i and IPSG j is 

keR, 

Similarly, at step 906, the cost (cfjk) of sending traffic along the static tunnel 
from each IPSG 206 to each CPE 222, and the current cost (fj) for using an IPSG 
5 node (j) 206 is determined for each customer /. In particular, the bandwidth 
requirement for traffic from IPSG j e Q to CPE k e R| is JVj* . Therefore, the 

bandwidth cost for each customer / between IPSG j and CPE k is d l 7 * = e'y*£V//* . 

Thus, the parameters c',j and dj k respectively denote the bandwidth cost of sending 
traffic from node i to node j, and from node j to node k. 
10 At step 908, the total bandwidth costs of the dynamic tunnels (Cci) are 

formulated as between the MAPs (p, of FIG. 6) and IPSGs (q, of FIG. 6). 
Specifically, the bandwidth cost of dynamic tunnels is C'ci = ]Tc'/y = J^sV ■ 

iePJeQ iePJeQ,keR, 

Further, at step 910, static tunnel bandwidth costs (C / c2) are formulated as 
between the IPSGs (g, of FIG. 6) and the CPEs (r* of FIG. 6). Specifically, the 
15 bandwidth cost of static tunnels is C l a = ^Td 1 ,* = J]c'j»jV • 

A service providers profits may be maximized by selecting optimal set of 
VPN customers to provision and selecting optimal IPSGs to provision each 
selected VPN customer. It is noted that profit ((/= Y V -d) is the difference 
between weighted revenue (Y \/) and cost (C), where revenue (\/) for a customer 
20 is a fixed value if the customer can be provisioned and Y is the relative weight on 

revenue compared to cost. 

The total cost (C 7 ) has several components, and as discussed above, such 
as determining the best set of IPSGs to provision each customer, which includes 
factoring in the cost of links in terms of bandwidth over which VPN tunnels are 
25 established, the cost of establishing each tunnel, the cost of provisioning each VPN 
customer on an IPSG, and redundancy in IPSG provisioning for fault tolerance. In 
other words, for customer /, for every MAP / in P and every CPE k in R u an IPSG j 
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in Q is selected to establish a unique dynamic tunnel between / and / and a shared 
static tunnel between j and k, such that the profit (U) is maximized. 

At step 912, the total tunnel bandwidth cost (Cc) is formulated. The total 
bandwidth cost is the sum of the dynamic tunnel bandwidth cost and the static 
5 tunnel bandwidth cost Cc = C?ci + & Cc2, where is the relative weight on the 
bandwidth cost of the static tunnel. Factors influencing the relative weight B on the 
static tunnel bandwidth cost include the cost of transporting data over core network 
over the cost over access network. 

At step 914, the current cost CV of provisioning a IPSG node (j) is 
10 formulated. The binary variable yy e {0,1} is 1 if IPSG j is provisioned for customer 
/ to send traffic to at least one of its CPEs, and it is 0 otherwise. The parameter fj is 
illustratively used as the current cost of using IPSG node /. For a given customer, 
at most one provision is considered at any IPSG. Therefore fj has a fixed valuej 
and the provisioning cost is CV= ^fjy'j ■• I 

j*Q j 

15 At step 916, the total cost for the customer is formulated. In particular, the 

total cost is Cl = Cc + cr CV, where a is the relative weight on the provision cost. 
Factors influencing the relative weight a on the provision cost include the 
importance of provision costs over bandwidth costs for the network service 
provider. 

20 At step 918, the profit is formulated. Generally, the profit is U = Y V - d. 

Without loss of generality, we assume that the revenue for each customer 
provisioned is the same. Naturally, both the revenue and cost are zero for each 
customer not provisioned. Revenue V 1 =ia/, where vJ represents whether customer 
/ g T is provisioned in the network. Therefore, the profit "L/ 1 " for provisioning the 

25 customer is U = f W - Cf, where Y is the relative weight on revenue compared to 
total cost. The weighting factor ^ essentially allows the network service provider to 
adjust price based on the total cost for the customer. 
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At step 920, given parameters, b' ik , ay, e' Jk , g i} , h' jk , fj, a, RJ , and, P C ap, 
integer variables s'^, binary variables yy, and w 1 are determined as the solution to 
the optimization problem formulation expressed as: 



10 



15 



20 



leT 



where 

C' =(c'cx+PC'c2)+oC'v 



C' = 



{iePJeQ JeQJceR, J. jeQ 



w' ={0,l},V/er 
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It is noted that equation (46) is an expanded version of equation (45). It is 
further noted that equation (50) reflects the fact that the bandwidth requirement of 
a customer only needs to be satisfied if the customer is provisioned. Conditions 
(51 through 54) are analogous to the single customer formulation discussed above. 
5 Conditions (55) and (56) are added to specify that customer / is provisioned on an 
IPSG only if some traffic for that customer is sent over that IPSG to a CPE. 
Condition (57) specifies that for a customer, if any traffic is sent through an IPSG, 
then the customer must be provisioned on that IPSG. Condition (58) is added to 
specify that the total number of provisions on each IPSG j cannot exceed its 

10 capacity P C ap- Condition (26) is added to make sure if customer / in not 
provisioned, y> is forced to be 0. 

In order to solve the integer programming problem of steps 820 and 920 of 
FIGS. 8 and 9, unit bandwidth costs ay, e' jk , and provision cost fj need to be j 
assigned appropriate values. The cost assignment can be adapted to fit the NSP's 

15 design objectives. This makes the formulation quite general and may be used for 
different scenarios. For example, suppose the NSP wants to satisfy a special 
requirement from a VPN customer that the users of this customer are not switched 
to a remote lightly loaded IPSG even if that reduces the total cost for the NSP. To 
be more specific, assume that the customer's requirement is that an MN on the 

20 east coast trying to access the corporate intranet on the east coast should not be 
redirected by the corresponding MAP to an IPSG on the west coast even if the total 
cost is minimized with this redirection. To take the constraint into account, we can 
either set the bandwidth capacity to 0 for the link (path) that connects the MAP to 
the IPSG on the west coast, or set the unit bandwidth cost on this link to infinity. 

25 For example, for a VPN customer, given MAP i in the east coast and IPSG j in the 
west coast, the input parameters may be assigned so that either a,y = 00 or g i} , = 0. 

When a single customer is considered, we have the option of setting 
provision cost to reflect the existing number of provisions at each IPSG. For 
example, we can use fj = cap/availj, where cap, is the capacity of IPSG j and availj 

30 is the number of available provisions left. This cost assignment will result in even 
distribution of the number of provisions per IPSG across all IPSGs. However, when 
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multiple customers are considered, the provision cost for different customers has to 
be the same to be a valid input to the integer programming program. Without loss 
of generality, we set fj ? = 1 for IPSG j for all customers. 

The number of hops from MAP node i to IPSG node j and from IPSG node j 
5 to CPE node k are assigned to unit bandwidth costs a,y and e) k , respectively, in 
order to reflect the fact that unit bandwidth cost for a link is proportional to the hop 
count on the path that is represented by that link. In order to illustrate the 
' relationship between bandwidth request and bandwidth capacity, the range of their 
values is limited. Bandwidth requests 6* is assigned to be random integers in the 

10 range [0, m], where m equals an integer greater than 1. Bandwidth capacities g,j is 
assigned to be random integers in the range [G, G + m], where G > 0 is the shift on 
the range of g,y relative to that of Similarly, bandwidth capacities are 
assigned to be random integers in the range [H, H + m], where H > 0 is the shift on 
the range of ht jk relative to that of b 1 *: The cost and capacity assignment phase 

15 accounts for customer specific requirements. Once this is performed, an integer 
programming package (i.e., for solving linear, mixed-integer, and quadratic 
programming problems), such as CPLEX, may be used to generate solution for 
specific parameter settings. 

Thus, the present invention provides a hierarchical architecture for providing 

20 network-based Mobile VPN services. The hierarchical architecture requires the 
Mobile Access Points (MAPs), which provide data connectivity to the mobile users 
to be physically separate from the IP Services Gateway (IPSG) that provides 
Mobile-VPN services. The hierarchical architecture enables more efficient use of 
the resources in the network. Based on the hierarchical architecture, the problem 

25 of provisioning Mobile-VPN customers on the IPSGs may be solved, such that the 
profit realized by a Network Service Provider (NSP) by providing such a service is 
maximized. The cost of providing Mobile-VPN service includes the cost of 
bandwidth incurred, as well as the cost of provisioning the customers on the 
IPSGs. The constraint faced by the NSP includes the bandwidth limitation on the 

30 links that connect the devices that form part of the service provider network, and 
the limitation on the maximum number of customers that can be provisioned on an 
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IPSG. An integer programming formulation is used to address this problem of 
provisioning Mobile-VPN customers. The formulation is general, and may be used 
to solve for practical configurations of the network topology and other parameters, 
and allows for various NSP requirements to be considered. The results provide 
5 insights into the design of Mobile-VPN architectures and services and illustrates 
the trade-offs that are involved in the design process. 

Although various embodiments that incorporate the teachings of the present 
invention have been shown and described in detail herein, those skilled in the art 
can readily devise many other varied embodiments that still incorporate these 
10 teachings. 
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